# Efficient deployment

A quantized version based on google/gemma-3-4b-it, using INT4 weight quantization and FP16 activation quantization to optimize inference efficiency

This is the MLX format version of the THUDM/GLM-4-32B-0414 model, processed with 4-bit DWQ quantization, suitable for efficient inference on Apple silicon devices.

Spec-T1-RL-7B is a high-precision large language model focused on mathematical reasoning, algorithm problem-solving, and code generation, and it performs excellently in technical benchmark tests.

Qwen3 30B A3B is a large language model that has undergone 8-bit quantization using the GPTQ method, suitable for efficient inference scenarios.

An INT4 weight quantization version based on openai/whisper-large-v3-turbo, supporting efficient audio-to-text tasks

Llama 2 is a 7B-parameter large language model developed by Meta, offering multiple quantization versions to accommodate different hardware requirements.

4-bit quantized version of QwQ-32B, optimized using Bitsandbytes technology, suitable for efficient inference in resource-constrained environments

This is a 4-bit quantized GPTQ model based on Meta Llama 3, quantized by Astronomer, capable of efficient operation on low-VRAM devices.

This is the ONNX format conversion of the MoritzLaurer/roberta-base-zeroshot-v2.0-c model, suitable for zero-shot classification tasks.